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ABSTRACT 

A  multi-person,  model-driven  experiment  has  been  designed  on  the  basis  of  a  mathematical 
model  of  distributed  tactical  decision  making.  Two  organizational  structures  are  used  in  the  investi¬ 
gation:  a  parallel  one  and  a  hierarchical  one.  The  performance  of  the  organization  is  measured  in 
terms  of  its  response  time  and  accuracy.  These  two  measures  represent  team  performance.  In  addi¬ 
tion,  the  cognitive  workload  of  decision  makers  (DMs)  during  the  execution  of  the  task  is  estimated 
because  bounded  rationality  imposes  a  limitation  on  the  human's  capability  for  processing  infor¬ 
mation  and  making  decision. 

The  results  show  that  interaction  among  DMs  compensates  for  differences  in  individual  per¬ 
formance  characteristics.  Individual  differences  have  more  influence  on  performance  in  the  organi¬ 
zation  in  which  DMs  have  more  autonomy  in  making  decisions  than  in  the  organization  in  which 
individual  decisions  are  coupled  with  the  decisions  of  other  organization  members.  When  available 
time  decreases,  time  pressure  is  introduced  in  the  organization  and  DMs  have  to  adjust  their  pro¬ 
cessing  rate.  The  experimental  results  confirm  a  hypothesis  which  predicts  that  with  decreasing 
available  time,  a  significant  degradation  of  performance  occurs  first  in  the  organization  which  has 
the  highest  minimum  feasible  workload. 
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ABSTRACT 

A  multi-person,  model-driven  experiment  has  been  de¬ 
signed  on  the  basis  of  a  mathematical  model  of  distributed  tacti¬ 
cal  decision  making.  Two  organizational  structures  are  used  in 
the  investigation:  a  parallel  one  and  a  hierarchical  one. 

The  performance  of  the  organization  is  measured  in  terms 
of  its  response  time  and  accuracy.  These  two  measures  represent 
team  performance.  In  addition,  the  cognitive  workload  of  deci¬ 
sion  makers  (DMs)  during  the  execution  of  the  task  is  estimated 
because  bounded  rationality  imposes  a  limitation  on  the  human's 
capability  for  processing  information  and  making  decision. 

The  results  show  that  interaction  among  DMs  compensates 
for  differences  in  individual  performance  characteristics. 
Individual  differences  have  more  influence  on  performance  in 
the  organization  in  which  DMs  have  more  autonomy  in  making 
decisions  than  in  the  organization  in  which  individual  decisions 
are  coupled  with  the  decisions  of  other  organization  members. 
When  available  time  decreases,  time  pressure  is  introduced  in 
the  organization  and  DMs  have  to  adjust  their  processing  rate. 
The  experimental  results  confirm  a  hypothesis  which  predicts 
that  with  decreasing  available  time,  a  significant  degradation  of 
performance  occurs  first  in  the  organization  which  has  the 
highest  minimum  feasible  workload. 


INTRODUCTION 

Distributed  decision  making  (DDM)  organizations  consist 
of  human  decision  makers  (DMs)  and  equipment,  structured  so 
as  to  accomplish  a  set  of  given  tasks.  In  the  past  few  years, 
several  research  efforts  have  been  started  to  explore  the 
characteristics  of  DDM  organizations  and  related  design  and 
evaluation  methodologies. 

One  such  effort  has  focussed  on  the  development  of  a  de¬ 
sign  and  evaluation  methodology  for  distributed  decision 
making  organizations  (Boettcher  and  Levis,  1983,  Tomovic  and 
Levis,  1984,  Jin,  Remy,  and  Levis,  1986,  Andreadakis  and 
Levis  1987,  and  Remy,  Levis,  and  Jin,  1988).  The  theoty  is 
based  on  the  mathematical  model  of  the  interacting  decision 
maker  with  bounded  rationality  (Boettcher  and  Levis,  1982). 
The  design  process  starts  with  the  generation  of  organizational 
structures  for  a  given  task.  Then,  the  procedures  and  protocols 
needed  to  perform  the  task  are  determined.  The  task  is 
characterized  by  the  input  uncertainty  that  needs  to  be  reduced 
in  order  to  make  decisions.  Information  theory  is  applied  to 
compute  the  entropy  of  the  input,  a  measure  of  uncertainty. 
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Then,  the  performance  of  the  organizational  design  can  be 
evaluated  (Levis,  1990).  While  the  model  and  the  methodology 
were  motivated  by  empirical  evidence  from  a  variety  of 
experiments  and  by  the  concept  of  bounded  rationality,  there 
were  no  direct  experimental  data  to  support  it.  An  experimental 
program  was  undertaken  to  test  the  theory  and  obtain  values  for 
the  model  parameters.  The  problems  under  study  are  those  that 
relate  organizational  structure  directly  to  performance,  as 
measured  by  accuracy  and  timeliness  and,  more  indirectly,  to 
cognitive  workload.  The  first  experiment  was  a  single  person 
experiment  designed  to  verify  the  existence  of  the  bounded 
rationality  constraint  (Louvet,  Casey,  and  Levis,  1988).  The 
experiment  provided  evidence  that  bounded  rationality  exists  and 
that  for  well-defined  tasks  the  onset  of  degradation  of 
performance  that  it  causes  can  be  predicted. 

To  explore  the  characteristics  of  organizational  perfor¬ 
mance,  multi-person  experiments  are  necessary.  The  study  re¬ 
ported  in  this  paper  shows  the  results  from  a  model-driven, 
multi-person  experiment  which  is  designed  to  investigate  the  ef¬ 
fect  of  organizational  structure  on  performance  of  DDM  organi¬ 
zations. 


METHODOLOGY 

In  the  experiments  involving  distributed  decision  making 
and  human  subjects,  the  major  difficulties  are  that  too  many 
variables  are  involved  and  too  many  uncertainties  exist  in  con¬ 
trolling  the  experiment.  Furthermore,  which  variables  should  be 
varied  and  over  what  range  become  critical  question  in  the 
experimental  design.  A  methodology  is  presented  in  this  section 
that  addresses  these  issues  and  guides  the  design  of  model- 
driven  experiments. 

Distributed  decision  making  organizations  operate  in  an 
environment  which  changes  dynamically.  A  change  in  the  envi¬ 
ronment  acts  as  a  stimulus  to  the  organization  which  in  turn 
senses  the  stimulus  and  processes  it  to  infer  what  the  situation 
is.  Then,  according  to  rules  and  procedures,  the  organization 
selects  a  response  to  the  environmental  change. 

In  many  distributed  decision  making  tasks,  time  con¬ 
straints  play  an  important  role.  For  a  given  task,  there  is  only  a 
limited  period  of  time  during  which  the  organizational  response 
will  be  effective.  This  time  period  is  called  the  window  of 
opportunity.  A  response  produced  too  early  or  too  late,  i.e„ 
outside  of  the  window  of  opportunity,  does  not  address  the 
requirement  of  the  task.  Therefore,  in  order  to  respond 
effectively,  the  tempo  of  operations  has  to  adjust  to  the  available 
time. 

Human  decision  makers  are  a  critical  component  in  DDM 
organizations.  Because  of  their  bounded  rationality  in 
processing  information  and  making  decisions,  organizational 
performance  degrades  if  human  DMs  are  overloaded. 

These  three  features  specify  the  context  in  which  a  model- 
driven  experiment  will  be  conducted. 
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There  are  four  major  stages  in  the  methodology  for  design¬ 
ing  model-driven  experiments:  (1)  Theoretical  analysis;  (2) 
Experimental  investigation;  (3)  Experimental  data  analysis;  and 
i4)  Comparison  of  the  theoretical  and  experimental  results.  In 
addition,  there  is  a  step  for  time  scale  calibration.  This  is  not 
considered  as  a  major  stage  because  a  small  separate  pilot 
experiment  is  sufficient  to  determine  the  time  range  for  a  given 
task.  Figure  1  shows  the  block  diagram  of  the  methodology. 


Issues  and  problems 


Figure  1  The  Methodology  for  Experimental  Design 

The  steps  in  each  of  the  four  design  stages  of  the  method¬ 
ology  are  described  in  the  following.  The  implementation  of 
these  steps  will  be  shown  through  an  application  in  the  next  sec¬ 
tion.  It  is  assumed  that  the  time  scale  calibration  has  been  done 
(see  Jin,  1990). 

Theoretical  Analysis 

A  task  which  will  be  performed  by  an  organization  is  se¬ 
lected.  This  task  should  reflect  the  problems  and  issues  to  be 
investigated.  The  same  task  is  used  both  in  the  theoretical  analy¬ 
sis  and  in  the  experiment. 

After  defining  the  task,  an  organization  is  designed  to 
carry  out  the  task.  The  design  of  an  organization  includes  the 
determination  of  the  protocol  and  the  procedures  to  be  used  in 
carrying  out  the  task.  The  organization  is  modeled  using  the 
Petri  Net  representation. 

The  evaluation  procedure  (CAESAR1),  which  includes 
simulating  the  operation  of  the  organizations  and  computing  the 
cognitive  workload  using  information  theory  (Boettcher  and 
Levis,  1983),  is  used  to  obtain  performance  measures  analyti¬ 
cally.  From  these  predictions,  the  Performance-Workload  locus 
(P-W  locus)  can  be  constructed.  The  characteristics  of  the  P-W 


1  CAESAR  (Computer  Aided  Evaluation  of  Systems  Architectures)  is  a 
noncommercial  set  of  programs  developed  at  MTT. 


locus  lead  to  the  generation  of  hypotheses  on  organizational  be¬ 
havior. 

Experimental  Investigation 

An  experiment  is  designed  to  test  the  hypotheses.  The 
complexity  of  the  DDM  organization  results  in  a  large  number  of 
parameters  and  in  much  uncertainty  regarding  their  values.  To 
determine  the  controlled  and  the  measured  parameters,  dimen¬ 
sional  analysis  is  applied  after  being  extended  to  include  the 
cognitive  aspects  of  distributed  decision  making  (Jin  and  Levis, 
1988).  Dimensional  analysis  (Hunsacker,  1947,  Gerhart,  1985) 
is  a  scientific  and  engineering  method  for  designing  experi¬ 
ments. 

In  order  to  carry  out  the  experiment,  the  range  of  the  con¬ 
trolled  parameters  needs  to  be  specified.  The  result  from  the  time 
scale  calibration  can  be  used  to  estimate  a  range  for  the  available 
time.  The  number  of  trials  for  each  value  of  controlled  parame¬ 
ters  must  also  to  be  determined. 

A  pilot  experiment  is  necessary  to  test  the  entire 
experimental  design.  Then,  the  actual  experiment  is  carried  out 
and  experimental  data  are  collected. 

Experimental  Data  Analysis 

In  this  stage,  the  collected  data  are  analyzed  and  pro¬ 
cessed  to  obtain  the  measures  of  performance.  The  procedures 
for  testing  the  hypotheses  are  determined.  These  procedures  are 
usually  statistical  ones.  To  apply  the  procedures,  the  hypotheses 
developed  in  the  theoretical  analysis  may  need  to  be  transformed 
into  an  explicit  form  which  can  be  tested  directly.  All  variables 
involved  in  the  hypothesis  testing  are  gathered  and  stored  in  an 
appropriate  format  for  the  test. 

Comparison  of  Theoretical  and  Experimental  Results 

This  is  the  last  stage  in  which  the  hypotheses  are  tested 
using  the  experimental  data.  The  theoretical  and  experimental  re¬ 
sults  are  compared  to  assess  the  model's  ability  to  predict  orga¬ 
nizational  behavior.  Final  conclusions  are  drawn. 

APPLICATION 

The  methodology  described  in  the  previous  section  is  used 
to  design  a  multi-person  experiment  for  investigating  the  effect 
of  organizational  structure  on  performance.  In  this  section,  the 
experiment  is  described.  The  first  stage  and  the  second  stage  of 
the  methodology  are  implemented. 

Description  of  the  Task 

Defense  in  a  naval  outer  air  battle  is  chosen  as  the  task  to 
be  performed  by  two  small  (three  person)  decision  making  or¬ 
ganizations.  The  objective  of  a  naval  outer  air  battle  is  to  monitor 
incoming  enemy  aircraft  and  deploy  interceptors  to  engage 
threats  so  as  to  prevent  the  enemy  from  entering  the  range  where 
missiles  can  be  fired  at  ships  in  the  battle  group. 

In  this  environment,  a  team  of  DMs  forms  an  outer  air  bat¬ 
tle  group  to  perform  the  above  task.  Specifically,  the  task  of  the 
DMs  is  to  detect  incoming  enemy  aircraft  ("threats");  find  out  the 
type  and  the  number  of  threats;  then  allocate  their  own  aircraft 
("resources")  to  intercept  the  threats. 

Figure  2  depicts  a  hypothetical  naval  outer  air  battle  envi¬ 
ronment.  The  carrier  is  at  the  center  of  the  circles.  Airborne 
warning  radar  aircraft  (E2C)  patrol  the  area  at  a  distance  Rp 
from  the  carrier.  Each  E2C  commands  several  squadrons  of 
interceptors,  which  can  intercept  directly  the  threats.  The  E2Cs 
are  equipped  with  passive  radar  (ESM)  and  active  radar.  ESM 
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receives  the  radar  transmission  of  other  aircraft  while  active 
radar  receives  the  reflection  of  its  transmission  by  other  objects. 
The  ESM  has  a  range  of  Ro  and  the  active  radar  has  a  range  of 
Ra  (  Ro  >  Ra  )•  Assume  that  the  range  of  the  enemy's  missiles  is 
Rm- 


Figure  2  A  Naval  Outer  Air  Battle  Environment 

ESM  has  a  larger  range  for  detecting  incoming  threats  but 
provides  less  specific  data  than  the  active  radar:  the  presence  and 
bearing  (direction  to)  of  the  threats.  The  active  radar  provides 
more  detailed  data  such  as  the  position  and  speed  of  a  threat. 
The  signature  of  an  aircraft  is  provided  by  ESM  when  the  threat 
is  closer.  An  emitter  signature  indicates  the  existence  of  an 
aircraft  with  its  corresponding  emitter. 

The  E2C  initially  operates  only  ESM  to  avoid  being  de¬ 
tected  by  the  enemy's  radar.  When  enemy  aircraft  approach  the 
E2C,  and  are  within  a  range  Ra,  the  E2C  turns  on  the  active 
radar.  When  all  information  (speed,  emitter  signature,  and  so 
on)  about  a  threat  is  available,  the  enemy  aircraft  can  be  identi¬ 
fied.  Correlation  between  the  emitter  signature  and  the  speed  of 
the  aircraft  can  be  used  to  classify  the  type  of  the  aircraft  with 
some  level  of  certainty. 

Based  on  the  assessment  of  incoming  threats,  the  E2C 
mission  commanders  allocate  resources  to  intercept  the  enemy 
aircraft.  The  resources  are  Tomcat  fighter  aircraft  (F14),  Homet 
fighter/attack  aircraft  (FI  8),  and  Prowler  aircraft  (EA-6B). 

There  are  situations  in  which  uncertainty  and  conflict  exist. 
For  example,  there  may  be  a  threat  detected  by  more  than  one 
E2C.  Then,  the  question  becomes  one  of  determining  who  is 
going  to  deal  with  it.  In  a  situation  like  this,  coordination  be¬ 
tween  organizational  members  is  necessary.  The  coordination  is 
done  through  communication.  In  addition,  the  E2C  mission 
commanders  may  have  to  communicate  with  the  outer  air  war¬ 
fare  commander  on  the  carrier  to  report  the  situation  or  to  ask 
him  to  launch  more  interceptors.  The  protocol  for  communica¬ 
tion  is  different  for  different  organizational  structures. 

When  an  enemy  aircraft  enters  its  missile  range  Rm  with¬ 
out  being  engaged  by  interceptors,  the  nature  of  the  task  changes 
to  the  inner  air  battle,  and  the  outer  air  battle  is  over.  Therefore, 
if  enemy  aircraft  are  not  intercepted  before  they  enter  the  inner 
air  battle  region,  the  outer  air  battle  defense  is  considered  to  have 
failed. 

The  above  description  of  the  naval  outer  air  battle  has  been 
abstracted  from  actual  operation  for  the  purpose  of  the  study. 
However,  the  abstraction  and  simplification  are  such  that  it  may 
not  reflect  the  reality  of  naval  operations. 

The  implementation  of  the  first  two  stages  of  the  method¬ 
ology  are  described  as  follows. 


Theoretical  Analysis 

To  perform  the  task  described  above,  two  three-DM  orga¬ 
nizational  structures  are  considered:  a  parallel  one  and  a  hierar¬ 
chical  one.  In  the  parallel  structure  (Fig.  3),  all  DMs  are  at  the 
same  level  of  authority.  They  are  working  together  in  coordina¬ 
tion.  In  the  hierarchical  structure  (Fig.  4),  authority  varies  with 
the  rank  of  a  DM,  that  is,  the  position  a  DM  holds  in  the  organi¬ 
zation.  DM2  in  Fig.  4  plays  a  supervisory  role  in  the  coordinat¬ 
ing  other  two  DMs.  In  both  structures,  the  task  is  the  same  and 
members  of  the  organization  have  to  act  as  a  team  to  perform  the 
task. 


Resource  Allocation 
Figure  3  Parallel  Organization 


Figure  4  Hierarchical  Organization 


The  procedure  is  as  follows.  When  threats  arc  detected, 
the  situation  has  to  be  assessed.  The  situation  assessment  (SA) 
function  provides  information  such  as  the  number  of  threats,  the 
position  and  speed  of  the  threats,  the  type  of  the  threats,  and  the 
number  of  aircraft  in  each  threat.  The  situation  assessment  in¬ 
volves  data  gathering  and  processing  because  some  of  the  in¬ 
formation  can  be  directly  obtained  from  the  observed  data  while 
other  information  is  available  only  after  the  raw  data  are  pro¬ 
cessed.  Depending  on  the  particular  situation,  communication 
may  be  required  after  the  situation  assessment  The  results  of  the 
communication  arc  processed  in  the  information  fusion  stage  for 
the  parallel  structure  and  in  the  command  interpretation  for  the 
hierarchical  structure.  The  last  stage  is  the  response  selection 
stage.  On  the  basis  of  the  fusion  data  or  command  interpretation 
results,  resources  can  be  allocated  to  counter  the  threats.  The 
task  is  completed  after  resources  are  allocated.  The  following 
paragraphs  briefly  describe  the  procedures  for  the  parallel  and 
hierarchical  organizations.  For  the  detailed  description,  see  Jin 
(1990). 
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Parallel  Organization 

In  the  parallel  organization,  the  defense  area  is  divided  into 
three  sectors.  Each  DM  is  an  E2C  mission  commander  and  is  re¬ 
sponsible  for  one  sector,  that  is,  this  DM  is  responsible  for  all 
threats  in  the  sector  and  only  this  sector.  There  are  overlap  areas 
between  the  sectors.  In  Fig.  5,  the  solid  straight  lines  in  the 
radar  screen  divide  the  defense  area  into  three  sectors.  The  area 
of  responsibility  of  a  DM  is  the  white  sector  bounded  by  two 
solid  straight  lines.  The  unshaded  areas  bounded  by  dotted  lines 
are  the  overlap  areas  between  the  sectors.  Each  DM  can  see  a 
pan  of  the  other  two  sectors.  Therefore,  the  area  that  can  be  seen 
by  a  DM,  defined  as  the  observation  area,  is  shown  by  the  area 
without  shading.  A  threat  in  the  shaded  area  cannot  be  seen  by 
and  is  out  of  the  region  of  responsibility  of  this  DM. 

It  should  be  clear  that  there  are  two  areas  for  each  DM  in 
which  the  responsibilities  are  different.  One  is  the  observation 
area,  the  white  area  in  Fig.  5,  which  includes  the  sector  and  the 
overlap  areas  of  other  two  adjacent  sectors.  Another  is  the  sec¬ 
tor,  bounded  by  solid  lines  in  Fig.  5.  The  threats  in  the  obser¬ 
vation  area  can  be  detected  and  the  information  about  these 
threats  can  be  obtained.  However,  a  DM  can  only  allocate  re¬ 
sources  and  intercept  the  threats  in  his  sector. 


The  threats  which  are  not  in  the  overlap  area  can  be  pro¬ 
cessed  without  exchanging  information  with  the  adjacent  DMs 
because  only  the  local  DM  can  observe  these  threats.  For  the 
threats  in  the  overlap  area,  partial  information  is  received,  there¬ 
fore,  coordination  with  other  DMs  in  the  team  is  necessary. 
Coordination  takes  place  through  communication.  In  the  real 
environment  of  an  outer  air  battle,  the  procedure  for  coordina¬ 
tion  is  quite  complex.  In  this  experiment,  the  procedure  is 
simplified  so  that  it  is  controllable  and  so  that  it  serves  the 
purpose  of  the  experiment. 

Hierarchical  Organization 

In  this  organization,  the  defense  area  is  divided  into  two 
sectors.  Two  DMs  are  E2C  mission  commanders  and  play  the 
role  of  subordinates.  Only  subordinates  can  observe  the  de¬ 
fended  area  directly.  The  third  DM  is  the  commander  on  the  car¬ 
rier,  he  performs  a  supervisory  role  and  coordinates  the  two 
sectors.  As  in  the  parallel  organization,  each  subordinate  is  re¬ 
sponsible  for  monitoring  and  intercepting  the  threats  in  one  sec¬ 
tor.  If  there  is  any  conflict,  that  is,  threats  are  in  the  overlap 
area,  subordinates  have  to  report  the  situation  from  their 
perspective  to  the  supervisor.  Then  they  have  to  wait  for 
commands  from  the  supervisor  before  prosecuting  this  threat. 
After  receiving  the  command,  a  subordinate  interprets  the 
commands  by  combining  the  local  situation  assessment  and  the 


commands  by  combining  the  local  situation  assessment  and  the 
commands,  then  makes  the  final  decision  on  the  resource 
allocation  (RS).  The  level  of  interaction  is  higher  in  the 
hierarchical  organization  than  in  the  parallel  organization. 

Different  algorithms  can  be  used  to  process  information 
and  make  decisions  in  the  SA  and  RS  stages.  The  basic  algo¬ 
rithms  are:  (a)  quick  estimation  with  attendant  risk  of  errors  and, 
(b)  accurate  acquisition  (probing)  but  with  time  delay.  Which 
one  should  be  chosen  depends  on  a  particular  situation,  i.e.,  on 
the  level  of  uncertainty,  on  the  time  available,  and  so  on.  The 
characteristics  of  the  two  algorithms  reflect  the  tradeoffs 
between  time  and  accuracy.  The  choice  of  different  algorithms 
indicates  the  strategy  used  during  the  execution  of  the  task. 

In  the  theoretical  analysis,  accuracy  J  and  workload  G  are 
computed  for  all  possible  strategies.  Then,  the  Performance- 
Workload  (J-G)  locus  is  constructed  to  predict  the  organizational 
performance.  For  the  detailed  description  of  the  computation, 
see  Jin  (1990). 

In  Fig.6,  the  projection  of  J-G  locus  on  the  workload 
plane  for  decision  makers  DMl  and  DM2  is  shown.  To  each  or¬ 
ganizational  strategy  corresponds  a  point  in  the  G1-G2  locus, 
G1  and  G2  are  the  workload  of  DMl  and  DM2  respectively.  For 
the  hierarchical  organization,  DMl  is  a  subordinate  while  DM2 
is  the  supervisor. 
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It  can  be  seen  from  Fig.  6  that  the  subordinate  (Gl)  in  the 
hierarchical  organization  has  the  highest  workload. 
Furthermore,  while  in  the  parallel  organization  the  workload  lo¬ 
cus  is  symmetric  -  the  two  DMs  shown  have  the  same  range  for 
task  workload  (Fig.  6b)  -  this  is  not  the  case  for  the  hierarchical 
organization  (Fig.  6a).  It  is  argued  that  in  the  hierarchical  orga¬ 
nization  with  a  protocol  requiring  close  interaction  among  DMs, 
when  one  DM’s  needed  task  processing  rate  exceeds  his  maxi¬ 
mum  processing  rate,  the  resulting  individual  degradation  in  per¬ 
formance  will  affect  organizational  performance. 

The  three  dimensional  pictures  in  Fig.  7  show  how  accu¬ 
racy  changes  with  workload  for  both  organizational  structures 
for  moderate  operating  times.  Hypotheses  will  be  generated  now 
by  interpreting  the  results  of  the  theoretical  analysis  and  aspects 
of  the  Performance-Workload  locus. 


requiring  less  workload.  This  method  of  coping  with  time 
pressure  works  until  the  maximum  rate  Fmax  is  attained  using 
the  strategy  with  the  least  required  workload.  Then  a  further 
decrease  of  Ta  will  result  in  a  rapid  degradation  of  performance 
since  no  strategy  is  available  to  do  the  task  completely.  The  DM 
will  fail  to  complete  the  task  and  may  make  random  errors  on  the 
portion  of  the  task  that  he  completes.  Therefore,  a  hypothesis 
can  be  formulated  as  follows. 

Hypothesis  1.  When  the  available  time  is  decreasing,  the 
organization  with  the  highest  minimum 
feasible  workload  for  a  given  set  of  strategies 
will  exhibit  a  performance  degradation  at  a 
larger  value  of  available  time  than  the 
organizations  which  have  lower  minimum 
workload. 


J 


Figure  7  (a)  Performance- Workload  Locus 
for  Hierarchical  Organization:  Moderate  Speed 


Because  of  bounded  rationality,  DMs  will  change  to 
strategies  with  less  workload  when  the  available  time  decreases. 
From  Fig.7,  it  can  be  seen  that  accuracy  decreases  with  an  in¬ 
crease  of  workload.  Given  that  in  this  experiment  the  minimum 
workload  strategy  yields  the  highest  performance,  there  is  no 
other  strategy  available  for  further  reduction  of  the  workload  to 
accommodate  a  shorter  available  time  when  a  DM  reaches  the 
maximum  processing  rate,  Fmax.  using  the  minimum  workload 
strategy.  Then,  the  ways  to  cope  with  the  situation  are  either  to 
reduce  the  number  of  communications  or  to  reduce  the  number 
of  threats  being  processed.  Since  the  objective  of  the  naval  air 
battle  is  to  process  completely  all  threats,  it  is  hypothesized  that 
a  decision  maker  will  omit  some  required  communication  in  fa¬ 
vor  of  processing  threats  in  his  own  sector.  While  this  strategy 
may  improve  individual  performance,  it  will  cause  a  rapid 
degradation  in  organizational  performance.  Consequently,  the 
onset  of  degradation  of  organizational  performance  should  occur 
at  the  same  time  that  the  number  of  communications  begins  to  be 
reduced. 

This  can  be  interpreted  as  selfish,  local  behavior.  Each 
DM,  under  pressure,  will  attempt  to  respond  to  the  threats  in  his 
sector  at  the  expense  of  organizational  performance.  Essentially, 
this  means  that  under  pressure,  individual  DMs  will  tend  to  de¬ 
couple  from  the  organization  by  reducing  coordination  and  op¬ 
erating  in  a  decoupled  mode.  If  this  were  not  the  case,  then 
degradation  of  performance  will  begin  before  reduction  in  com¬ 
munications  and  the  latter  will  be  more  gradual  than  performance 
degradation.  For  this  argument,  the  following  hypothesis  is 
formulated. 

Hypothesis  2  Since  the  minimum  workload  strategy 
yields  highest  performance,  under  in¬ 
creased  time  pressure  decision  makers 
will  reduce  communications 
(coordination)  with  an  attendant  reduc¬ 
tion  in  organizational  performance. 

These  two  hypotheses  will  be  tested  by  the  experiment. 


Figure  7  (b)  Performance-Workload  Locus 
for  Parallel  Organization:  Moderate  Speed 

Hypotheses 

To  generate  hypotheses  form  the  model  predictions  of  or¬ 
ganizational  performance  shown  in  Figs.  6  and  7,  let  us 
consider  what  will  happen  when  the  available  time  decreases. 

When  Ta  decreases,  the  processing  rate  F  increases  while 
the  task  workload  is  kept  constant  (Jin,  1990).  If  Ta  decreases 
continuously  until  the  processing  rate  reaches  the  maximum 
value  Fmax.  further  decrease  of  Ta  will  force  a  reduction  of 
workload  which  is  accomplished  by  the  DM  selecting  a  strategy 


Experiment 

The  experiment  is  based  on  a  simulation  of  a  naval  outer 
air  battle  (OAB)  environment.  Human  operators  interact  with  the 
computer  simulation  of  the  OAB  and  make  decisions  for  execut¬ 
ing  the  task.  The  task  is  divided  into  subtasks  which  are  carried 
out  by  different  DMs  in  the  organization.  Each  DM  has  a  display 
to  observe  the  OAB  situation.  The  display  consists  of  a  simu¬ 
lated  radar  screen  in  which  threats  are  displayed,  a  board  for 
numerical  values,  a  window  showing  the  resource  status,  and  a 
communications  window  displaying  the  incoming  and  outgoing 
messages.  In  addition,  several  buttons  are  present  that  can  be 
pressed  by  the  DM.  The  detailed  description  of  the  display  is 
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given  in  (Jin,  1990).  The  interaction  between  decision  makers 
necessary  for  completing  the  task  is  realized  by  communication 
through  computer  networks. 

To  investigate  the  effects  of  organizational  structure  on 
performance,  two  different  structures  are  used  in  the 
experiment.  In  each  organizational  structure,  decision  makers 
are  organized  into  teams  and  then  these  teams  perform  the  task. 
The  performance  of  the  teams  is  measured. 

Dimensional  analysis  was  extended  to  include  cognitive 
aspect  and  applied  to  determine  the  controlled  variables  and  their 
ranges  in  the  experiment  (Jin,  1990).  Two  dimensionless  vari¬ 
ables,  time  ratio  and  communication  ratio,  were  suggested  by 
dimensional  analysis.  The  time  ratio  is  defined  as  the  ratio  of  the 
actual  processing  time  (Tf)  and  the  available  time  (Ta)  to  do  a 
task.  The  communication  ratio  is  defined  as  the  ratio  of  the 
actual  number  of  communications  (Nc)  and  the  task-required 
number  of  communications  (Nrc).  The  task-required  number  of 
communications  depends  on  the  procedure  and  protocol  of  an 
organization.  Later  in  this  paper,  these  ratios  are  used  to 
characterize  organizational  behavior. 

The  experiment  was  run  with  fifteen  teams  in  which  each 
team  has  two  human  DMs  and  one  DM  role  played  by  a  com¬ 
puter.  There  were  30  human  subjects.  Among  the  subjects.  28 
were  students  and  two  were  MIT  employees.  Seven  of  the  28 
students  were  graduate  students,  nineteen  were  undergraduate 
students,  and  two  were  middle  school  students.  Both  MIT  em¬ 
ployees  had  college  or  graduate  degrees.  The  teams  were  num¬ 
bered  3  to  17;  teams  1  and  2  were  the  ones  that  participated  in 
the  pilot  experiment.  The  experiment  was  carried  out  during  the 
winter  Independent  Study  Period  1990  at  MIT,  which  is  about 
one  month  long. 


EXPERIMENTAL  RESULTS 

In  this  section,  the  observations  from  the  experimental  data 
are  discussed.  Two  important  results  are  drawn  from  these  ob- 
servauons:  ta)  interaction  between  orgamzauonal  members  Com 
pensated  for  individual  differences;  and  (b)  coordination  by  the 
supervisor  in  the  hierarchical  organization  reduced  the  variance 
of  organizational  performance.  The  hypotheses  generated  from 
the  model  are  then  tested. 

Organization  versus  Individuals 

When  the  amount  of  work  required  by  a  task  is  such  that  it 
cannot  be  handled  by  a  single  person,  an  organization  is  formed. 
A  properly  designed  organization  will  maintain  performance  at  a 
desired  level.  Furthermore,  organizational  performance  should 
not  be  sensitive  to  variations  in  individual  skills.  Figures  8  and 
9  show  a  comparison  of  the  standard  deviation  of  the  accuracy 
measure  for  each  organization  and  for  individual  DMs  in  the  two 
structures. 
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Figure  8  Standard  Deviation  of  J  for  Teams  and  Individuals 


When  the  available  time  is  long  enough  to  do  the  task,  the 
standard  deviations  between  the  teams  and  individuals  are  very 
close  because  the  task  can  be  completed  accurately  and  the  error 
is  random.  However,  when  time  is  decreased,  individual  differ¬ 
ences  in  skills,  experience,  and  capabilities  are  revealed.  The 
standard  deviation  of  individual  performance  is  high.  On  the 
other  hand,  the  organizational  performance  is  more  stable.  This 
observation  can  be  explained  as  follows. 
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Figure  9  Standard  Deviation  of  J  for  Teams  and  Individuals 


An  organization  is  designed  so  that  the  task  is  divided  into 
subtasks  and  allocated  to  all  organizational  members.  Each  of 
the  members  in  the  organization  will  interact  with  other  or¬ 
ganizational  members  and  contribute  a  part  of  the  effort  to 
perform  the  task.  The  decisions  of  one  member  will  affect  the 
decisions  of  the  other.  Therefore,  compensatory  behavior 
between  the  organizational  members  reduces  the  variance  of 
organizational  performance.  As  a  consequence,  organizational 
performance  is  less  sensitive  to  individual  difference.  Both 
Fig.  8  and  Fig.  9  show  that  the  standard  deviation  of  the 
accuracy  measure  for  teams  is  much  smaller  than  that  of 
individuals  during  the  fast  tempo  of  the  operations. 

Although  the  actual  standard  deviation  is  different  in  the 
different  structures,  the  phenomenon  is  observed  for  both 
structures.  Therefore,  it  can  be  concluded  that  team  work  re¬ 
duces  the  effect  of  individual  differences  on  performance. 

Effects  of  Organizational  Structures 

Figure  10  shows  the  comparison  of  the  standard  deviation 
of  the  accuracy  J  for  the  two  structures.  The  standard  deviations 
are  computed  across  15  teams  for  both  organizational  structures. 
It  can  be  seen  from  Fig.  10  that  for  most  values  of  Ta,  J  has 
smaller  standard  deviation  in  the  hierarchical  organization  than  in 
the  parallel  organization.  This  implies  that  J  of  tne  hierarchical 
organization  is  more  robust  with  respect  to  the  individual  differ¬ 
ences  than  that  of  the  parallel  organization. 

The  difference  in  the  standard  deviation  reflects  the  organi¬ 
zational  effects.  As  already  stated,  the  interaction  level  in  the 
hierarchical  organization  is  higher  than  that  in  the  parallel  organi¬ 
zation.  In  terms  of  making  decisions,  DMs  in  the  parallel  organi¬ 
zation  have  more  "freedom"  to  choose  what  to  do  than  those  in 
the  hierarchical  organization.  Therefore,  it  is  expected  that  indi¬ 
vidual  difference  will  have  more  influence  on  performance  in  the 
parallel  organization.  On  the  other  hand,  the  interactions  in  the 
hierarchical  organization  restrict  the  choices  of  the  decision  mak¬ 
ers  and  couple  individual  decisions  with  the  decisions  of  other 
organization  members.  As  a  result,  individual  characteristics 
tend  to  be  suppressed  in  the  organizational  performance. 
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Figure  10  Standard  Deviation  of  J  for  the  Two  Structures 


Testing  the  Hypotheses 

Hypothesis  1  predicts  that  the  organization  with  the 
highest  minimum  workload  will  show  performance  degradation 
prior  to  those  organizations  which  have  lower  minimum 
workload.  From  Figs.  6  and  7,  it  is  shown  that  the  subordinates 
in  the  hierarchical  organization  have  the  highest  minimum 
workload. 

Let  T*h  and  T*p  be  the  available  times  at  which  the  per¬ 
formance  of  the  hierarchical  organization  and  the  parallel  organi¬ 
zation  degrade  sharply.  Hypothesis  1  can  be  expressed  as  fol¬ 
lows. 

HO:  T'h  >  T'p;  Hypothesis  1  is  accepted; 

HI:  T'h  <  T*p;  Hypothesis  1  is  rejected. 

To  test  Hypothesis  1,  the  mean  values  of  the  critical  avail¬ 
able  times,  which  is  the  available  time  at  which  performance  be¬ 
gins  to  degrade  rapidly,  for  both  organizational  structures,  need 
to  be  computed  from  li.e  experimental  data. 

Accuracy  versus  available  time  (J-Ta  plot)  are  plotted  for 
each  team.  Figure  1 1  is  a  such  plot  for  one  team.  The  observa¬ 
tion  from  the  J-Ta  plots  indicates  that  there  exists  a  region  in 
which  J  stans  to  degrade  rapidly.  To  estimate  the  Ta  at  which 
»uch  degradation  occurs,  a  piece-wise  linear  fit  is  perfumed. 
Two  asymptotes  are  found.  The  intersection  point  of  the  two 
asymptotes  can  be  used  to  estimate  the  T*  values. 
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Figure.  1 1  Accuracy  and  Available  Time  for  a  Team: 
Team  #10 


To  find  the  asymptotes,  the  Least  Square  (LS)  fit  is  used. 
As  an  example,  Fig.  12  shows  the  original  curves  and  the 
asymptotes  found  by  using  the  LS  fit. 

Table  1  show  the  results  computed  from  the  data  of  15 
teams.  The  mean  value  of  Th*  is  62.49  seconds  and  Tp*  is 
58.55  seconds.  Hypothesis  1  is  confirmed. 

Team  #8 


Table  1  Ta*  for  Both  Organizational  Structures 
(unit:  in  seconds) 


Hierarchical 

Parallel 

Mean 

6 515 

503 

$t.  Dev. 

I7TM 

TTTg 

If  Hypothesis  2  is  correct,  the  rapid  reduction  in  the  num¬ 
ber  of  communications  and  rapid  reduction  of  organizational 
performance  will  occur  at  the  same  time.  Because  of  the 
difference  in  structures  and  protocols  in  the  hierarchical 
organization  and  the  parallel  organization,  the  required  number 
of  communications  by  the  task,  Nrc,  is  very  different. 
Therefore,  it  is  necessary  to  take  Nrc  into  consideration.  The 
communications  ratio.  nc,  is  constructed  by  normalizing  the 
actual  number  of  communications,  Nc,  by  the  task-required 
number  of  communications,  Nrc- 


Correspondingly,  the  time  ratio,  t,  is  used  in  the  test, 


Let  t*c  denote  the  time  ratio  at  which  the  number  of  com¬ 
munications  reduces  rapidly  and  t*j  denote  the  time  ratio  when 
the  performance  starts  to  drop  significantly.  Then  Hypothesis  2 
can  be  expressed  as 

HO:  t*c  =  t'j;  Hypothesis  2  is  accepted; 

HI:  t*c*t'j;  Hypothesis  2  is  rejected. 

To  test  this  hypothesis,  two  relations  need  to  be  analyzed: 
1)  the  relation  between  the  communications  ratio  and  time  ratio 
and,  2)  the  relation  between  accuracy  and  time  ratio.  When  the 
critical  time  ratio  corresponding  to  the  communications  ratio  is 
determined,  it  can  be  compared  with  the  critical  time  ratio  found 
from  the  relation  between  accuracy  and  time  ratio  to  test 
Hypothesis  2. 
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Figure  13  is  a  plot  of  communication  ratio  versus  the 
complement  of  the  time  ratio  ( 1  -  Tf/Ta)  for  a  team.  To  find  the 
value  of  t*c.  a  piece- wise  linear  fit  is  performed.  Two  asymp¬ 
totes  are  found.  The  intersecnon  point  of  the  two  asymptotes  can 
be  used  to  estimate  the  t*c  value. 

To  find  the  asymptotes,  the  Least  Square  (LS)  fit  is  used. 
As  an  example.  Fig.  14  shows  the  original  curves  and  the 
asymptotes  found  by  using  the  LS  fit  for  the  hierarchical  organi¬ 
zation  of  the  team  in  Fig.  1 3. 

Team  #13 


Figure  13  Communications  Ratio  versus  Time  Ratio 


Team  #13:  Hierarchical:  (Tf/Ta)*=0.95 


Team  #13:  Hierarchical.  (Tf/Ta)*=0.95 


Table  2  t*  Values  for  J  and  nc 
(Unit:  in  Seconds) 


Hierarchical 

F*arallel 

t*J 

t*j 

t*c 

Mean 

0.89 

0.89 

0.89 

0.94 

St.  Dev. 

0.06 

0.06 

0.10 

0.04 

In  Table  2.  both  of  t*’s  vary  over  a  narrow  range.  The 
standard  deviations  of  t*j  are  0.06  and  0. 1  for  the  hierarchical 
organization  and  the  parallel  organization,  respectively,  and  0.06 
and  0.04  for  t*c. 

For  the  hierarchical  organization.  Table  2  shows  that 
mean  tj  =  mean  tj  =  0.89 

Therefore,  Hypothesis  2  is  confirmed  for  the  hierarchical  orga¬ 
nization.  However,  for  the  parallel  organization. 


Figure  14  Asymptote  for  nc  and  (1 -Tf/Ta) 

A  similar  method  is  used  to  find  the  critical  time  ratio  for 
accuracy.  Figure  15  shows  the  plot  of  accuracy  J  versus  the 
complement  of  the  time  ratio:  Fig.  16  shows  the  asymptotes  for 
the  hierarchical  organization.  Table  2  displays  the  mean  values 
and  standard  deviations  of  t*j  and  t*c  for  both  structures. 
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Figure  15  Accuracy  and  Time  Ratio 


mean  tj  =  0.89  and  mean  tj  =  0.94 

Therefore,  Hypothesis  2  is  disproved  for  the  parallel  or¬ 
ganization.  Since  t*c  is  larger  than  t*;,  it  follows  that  when  the 
number  of  communication  drops  significantly,  performance 
does  not  yet  degrade  rapidly.  The  explanation  is  the  following. 

As  discussed  when  Hypothesis  2  was  formulated,  there 
are  two  ways  to  reduce  load  in  the  absence  of  a  strategy  that 
permits  completion  of  the  task:  to  reduce  the  number  of  com¬ 
munications  or  to  proces:  fewer  threats.  There  are  a  few  teams 
that  chose  the  second  way  to  cope  with  time  pressure  when  op¬ 
erating  in  parallel.  In  addition,  recall  that  the  objective  is  to  pro¬ 
cess  completely  all  threats.  When  time  pressure  is  very  high,  the 
DMs  attempted  to  reduce  the  number  of  communications  in 
order  to  complete  the  threats  in  their  own  sector.  However, 
communications  in  the  parallel  organization  affect  only  a  small 
portion  of  the  total  threats  that  need  to  be  processed.  Therefore, 
a  partial  reduction  in  communications  does  not  affect 
organizational  performance  significantly. 

CONCLUSIONS 

The  study  is  focussed  on  the  effect  of  organizational 
structure  on  performance  of  decision  making  teams.  There  are 
two  aspects  which  will  affect  organizational  performance.  One  is 
the  task  attributes.  Another  is  the  information  processing  and 


decision  making  ability  of  the  organizational  members.  Task  at¬ 
tributes  change  the  operating  conditions  in  which  a  DDM  organi¬ 
zation  operates.  Individual  differences  of  DMs  result  in  a  van- 
ability  of  organizational  performance.  Both  aspects  were  stud¬ 
ied.  The  results  are  as  follows. 

The  main  task  attributes  that  was  changed  was  the 
available  time.  When  Ta  decreases,  time  pressure  is  introduced 
in  the  organization  and  DMs  have  to  adjust  their  processing  rate. 
However,  when  the  processing  rate  reaches  its  maximum  value, 
further  decrease  of  the  available  time  causes  transition  to  lower 
workload  strategies  until  the  minimum  workload  strategy  is 
reached.  When  no  strategy  is  available  to  reduce  the  workload  in 
order  to  accommodate  a  shorter  available  time,  rapid  degradation 
in  performance  occurs.  The  expenmental  results  confirm  a  hy¬ 
pothesis  which  predicts  that  with  decreasing  available  time,  a 
significant  degradation  of  performance  occurs  first  in  the  organi¬ 
zation  which  has  the  highest  minimum  workload. 

When  individual  performance  and  team  performance  are 
compared,  the  result  shows  that  organizational  performance  is 
more  predictable  than  individual  performance.  The  reason  is  that 
the  interaction  among  DMs  compensates  for  differences  in  indi¬ 
vidual  performance  characterisncs. 

Individual  difference  has  more  influence  on  performance 
in  the  parallel  organization.  On  the  other  hand,  the  interactions  in 
the  hierarchical  organization  restrict  the  choices  of  the  decision 
makers  and  couple  individual  decisions  with  the  decisions  of 
other  organization  members.  As  a  result,  individual  characteris¬ 
ncs  tend  to  be  suppressed  in  the  organizational  performance. 

In  addition  to  the  results  shown  in  the  previous  section, 
the  time  ratio  introduced  by  dimensional  analysis  provides 
useful  information  on  the  determination  of  the  available  time 
organization  design.  The  critical  time  ratio  at  which 
organizational  performance  degrades  rapidly  implies  the  shortest 
available  time  for  doing  a  task.  This  ratio,  together  with 
information  from  the  time  calibration,  can  be  used  to  specify  the 
range  of  available  time  for  a  given  task  in  a  new  design. 

REFERENCE 

Andreadakis.  S.  K.  and  A.  H.  Levis  (1987).  "Accuracy  and 
Timeliness  in  Decision-making  Organizations,"  Proc. 
10th  1FAC  World  Congress.  Pergamon  Press,  New 
York. 

Boettcher,  K.  L.  and  A.  H.  Levis  (1982).  "Modeling  the 
Interacting  Decisionmaker  with  Bounded  Rationality," 
IEEE  Trans,  on  Systems.  Man.  and  Cybernetics.  Vol. 
SMC- 12,  No.  3. 

Boettcher,  K.  L.  and  A.  H.  Levis  (1983).  "Modeling  and 
Analysis  of  Teams  of  Interacting  Decisionmakers  with 
Bounded  Rationality,"  Automatica,  Vol.  19,  No.  6. 

Gerhart,  P.  M.  and  R.  J.  Gross  (1985).  Fundamentals  of  Fluid 
Mechanics.  Addison- Wesley,  Reading,  M.A. 

Hunsaker,  J.  D.  and  B.  G.  Rightmire  (1947).  Engineering 
Applications  of  Fluid  Mechanics.  McGraw-Hill,  New 
York. 

Jin,  V.  Y.  (1990).  'Effect  of  Organizational  Structure  on 
Performance  of  Decision  Making  Teams",  LIDS-TH- 
1976,  PhD  Thesis,  Laboratory  for  Information  and 
Decision  Systems,  MIT,  Cambridge,  MA. 

Jin,  V.  Y.  and  A.  H.  Levis  (1988).  "Command  and  Control 
Experiment  Design  Using  Dimensional  Analysis,"  Proc. 
Command  and  Control  Research  Symposium ,  Monterey, 


CA,  LIDS-P-1787,  Laboratory  for  Information  and 
Decision  Systems,  MIT,  Cambridge,  MA. 

Jin,  V  Y.,  A.  H.  Levis,  and  P.  A.  Remy  (1986).  "Delays  in 
Acyclical  Distributed  Decisionmaking  Organizations," 
Proc.  IF  AC  Symposium  on  Large  Scale  Systems:  Theory 
and  Applications.  Zurich,  Switzerland,  August  1986 

Levis,  A.  H.  (1990).  “Distributed  Intelligent  Systems.”  Class 
notes  for  Course  6.291  in  Department  of  Electrical 
Engineering  and  Computer  Science,  MIT. 

Levis,  A.  H.  and  K.  L.  Boettcher  (1983).  "Decisionmaking 
Organizations  with  Acyclical  Information  Structures," 
IEEE  Trans,  on  Systems.  Man,  and  Cybernetics,  Vol. 
SMC- 13,  No. 3. 

Lou  vet,  A.  C.,  J.  T.  Casey,  and  A.  H.  Levis  (1988). 
"Experimental  Investigation  of  Bounded  Rationality 
Constraint,"  in  Science  of  Command  and  Control,  S.  E. 
Johnson  and  A.  H.  Levis,  Eds.  AFCEA  International 
Press,  Washington,  DC. 

Remy,  P.  A.,  A.  H.  Levis,  and  Jin,  V.  Y.  (1986).  "On  the 
Design  of  Distributed  Organizational  Structures," 
Automatica,  Vol.  24,  No.  1 

Tomovic,  M.  M.  and  A.  H.  Levis  (1984).  "On  the  Design  of 
Organizational  Structures  for  Command  and  Control,"  in 
Proc.  7th  MOT/ONR  Workshop  on  C3  Systems. 
LIDS-P-1419,  MIT,  Cambridge,  MA. 


9 


